IRIX Base Documentation 2002 November

home *** CD-ROM | disk | FTP | other *** search

/ IRIX Base Documentation 2002 November / SGI IRIX Base Documentation 2002 November.iso / usr / share / catman / p_man / cat5 / realtime.z / realtime

Wrap

Text File | 2002-10-03 | 26.6 KB | 529 lines

rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) NNNNAAAAMMMMEEEE _rrrr_eeee_aaaa_llll_tttt_iiii_mmmm_eeee, _ssss_cccc_hhhh_eeee_dddd_uuuu_llll_eeee_rrrr - introduction to real time and scheduler facilities DDDDEEEESSSSCCCCRRRRIIIIPPPPTTTTIIIIOOOONNNN The IRIX operating system provides a rich set of real-time programming features that are collectively referred to as the REACT extensions. This document introduces the components of REACT, including: bounded response time, clocks, timers, signals, virtual memory control, asynchronous I/O, POSIX threads, scheduling policies, realtime priority band, processor isolation, process binding, interrupt redirection and kernel thread placement. _BBBB_oooo_uuuu_nnnn_dddd_eeee_dddd _RRRR_eeee_ssss_pppp_oooo_nnnn_ssss_eeee _TTTT_iiii_mmmm_eeee A real-time system provides bounded and usually fast response to specific external events, allowing applications to schedule a particular thread to run within a specified time limit after the occurrence of an event. IRIX guarantees deterministic response of one millisecond on certain uni-processor systems. This real-time strategy guarantees the highest priority thread will execute within one millisecond from the time it was made runnable. On certain multi-processor machines (OCTANE, Origin200, Origin2000, Onyx2, Origin300, Origin3000 series, and Onyx3), the one millisecond bounded response time guarantee is controlled by the systune variable _r_t_c_p_u_s. rtcpus represents a threshold at which the scheduler functionality that is required to meet this guarantee is enabled. The threshold is based on the number of physical cpus in the system. If rtcpus is set greater than or equal to the number of physical processors, the bounded response guarantee is enabled. If rtcpus is set below the number of physical processors in the machine, the bounded response time guarantee is NOT enabled. The default value for rtcpus is 0, which means that by default, the guarantee is not enabled. In order to enable the guarantee, rtcpus must be set equal to or greater than the number of cpus in the system. As an example, consider a four processor system. If rtcpus is set at a value between 0 and 3 (inclusive), the real-time guarantee is not enabled. If rtcpus is set at 4 or greater, the real- time guarantee is enabled. Note that enabling the real-time guarantee may cause overall system performance to degrade. Real-time applications requiring a lower latency guarantee can use the multi-processor real-time strategy to obtain a deterministic response of 50 microseconds on Origin3000, Origin300, and Onyx3 series machines and 100 microseconds on Origin2000, Origin200, and Onyx2 series machines. This strategy typically consists of having one processor service unpredictable loads, such as interrupts and system daemons, and the other processor(s) servicing high-priority real-time jobs. PPPPaaaaggggeeee 1111 rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) _CCCC_llll_oooo_cccc_kkkk_ssss In order to perform event timing, IRIX provides the POSIX 1003.1b _c_l_o_c_k__g_e_t_t_i_m_e(2) interface. This interface can be used to access various system clocks, including: the real-time clock, and a low overhead free running hardware counter. _TTTT_iiii_mmmm_eeee_rrrr_ssss IRIX implements both BSD itimers and POSIX 1003.1b timers. POSIX timers are recommended for real-time application development, as they provide the highest resolution and flexibility (see _t_i_m_e_r__c_r_e_a_t_e(3c)). Timer expiration interrupts are dispatched to IRIX interrupt threads for handling. The priority at which these threads are scheduled is determined by the scheduling policy and priority of the thread which set the timer: If the thread setting the timer is running with a timeshare scheduling policy, then the associated interrupt thread will be scheduled at realtime priority one. If the thread setting the timer is running with a real-time scheduling policy, then the priority of the associated interrupt thread will be the priority of the setting thread plus one. Priority 255, being the maximum real-time band priority, is an exception. If the thread setting the timer is running at priority 255, then the interrupt thread will also be scheduled at priority 255. Hence, real-time applications depending on system services shouldn't use priority 255 (see the _RRRR_eeee_aaaa_llll_----_TTTT_iiii_mmmm_eeee _PPPP_rrrr_iiii_oooo_rrrr_iiii_tttt_yyyy _BBBB_aaaa_nnnn_dddd Section below). Once the timer expires, the interrupt thread will be scheduled ahead of the thread which set the timer. _SSSS_iiii_gggg_nnnn_aaaa_llll_ssss IRIX supports the full semantics of both BSD and AT&T signals. In addition IRIX has implemented the POSIX 1003.1b queued signals which provide signal priorities and for queuing of signals such that exactly as many signals are received as were sent (see _s_i_g_q_u_e_u_e(3)). IRIX does not guarantee the latency of signal delivery. _MMMM_eeee_mmmm_oooo_rrrr_yyyy _LLLL_oooo_cccc_kkkk_iiii_nnnn_gggg A real-time application can avoid the overhead of page fault processing under IRIX by locking ranges of its text and data into memory. The POSIX _m_l_o_c_k_a_l_l(3c) system call can be used to lock down a process's entire virtual address space. Since it is not always desirable to lock down the entire virtual address space, IRIX provides the following system calls to lock and unlock a specified range of addresses in memory: _m_p_i_n(2)/_m_u_n_p_i_n(2) and _m_l_o_c_k(3c)/_m_u_n_l_o_c_k(3c). The major difference between the two sets is that mpin/munpin maintains a per page lock PPPPaaaaggggeeee 2222 rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) counter and mlock/munlock does not. Developers should choose the set that best suits their application and stick with it, as mixing the interfaces may result in unexpected behavior. _AAAA_ssss_yyyy_nnnn_cccc_hhhh_rrrr_oooo_nnnn_oooo_uuuu_ssss _IIII_////_OOOO IRIX implements the POSIX 1003.1b interface to asynchronous I/O. Using this facility a programmer can queue a read or write request to a device and optionally receive a queued signal when the request completes. The _r_e_a_d() or _w_r_i_t_e() call will return when the request is queued rather than blocking the process pending completion of the I/O. Optionally, process priority can be used to establish the order in which queued requests are completed. _PPPP_OOOO_SSSS_IIII_XXXX _TTTT_hhhh_rrrr_eeee_aaaa_dddd _SSSS_cccc_oooo_pppp_eeee POSIX threads (pthreads) supports both process and system scope threads. System scope threads enable pthread applications to obtain predictable scheduling behavior on a system level by using the kernel scheduler directly, bypassing the user-level pthread scheduler. For more information about the pthread scheduling model, see _p_t_h_r_e_a_d(5). _TTTT_iiii_mmmm_eeee_ssss_hhhh_aaaa_rrrr_eeee _SSSS_cccc_hhhh_eeee_dddd_uuuu_llll_iiii_nnnn_gggg IRIX has an earnings-based scheduler for timeshare threads. Processes earn cpu microseconds of time base on their proportional share of the system. Their share of the system, and thus the rate at which they accumulate earnings, is determined by their nice value. While timeshare threads are not priority scheduled, they do have an independent timeshare priority band to represent _n_i_c_e(2) values. This band ranges from a low priority of 1 to a high priority of 40. A change in either the timeshare priority or the nice value results in a corresponding change to the nice value or timeshare priority respectively. Timeshare threads which are not the beneficiaries of priority inheritance are never scheduled ahead of real-time threads. _BBBB_aaaa_tttt_cccc_hhhh _SSSS_cccc_hhhh_eeee_dddd_uuuu_llll_iiii_nnnn_gggg Refer to _m_i_s_e_r(5). _RRRR_eeee_aaaa_llll_----_TTTT_iiii_mmmm_eeee _SSSS_cccc_hhhh_eeee_dddd_uuuu_llll_iiii_nnnn_gggg IRIX supports the POSIX 1003.1b real-time scheduler interfaces, including: _s_c_h_e_d__s_e_t_s_c_h_e_d_u_l_e_r(2) and _s_c_h_e_d__s_e_t_p_a_r_a_m(2). These interfaces provide privileged applications with the control necessary for managing the cycles of the system processor(s). Real-time scheduling policies, such as round-robin and first-in-first-out, may be selected along with a real-time priority. PPPPaaaaggggeeee 3333 rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) _RRRR_eeee_aaaa_llll_tttt_iiii_mmmm_eeee _PPPP_rrrr_iiii_oooo_rrrr_iiii_tttt_yyyy _BBBB_aaaa_nnnn_dddd A real-time thread may select one of a range of 256 priorities (0-255) in the real-time priority band, using POSIX interfaces _s_c_h_e_d__s_e_t_p_a_r_a_m() or _s_c_h_e_d__s_e_t_s_c_h_e_d_u_l_e_r(). The higher the numeric value of the priority the more important the thread. Developers must consider the needs of the application and how it should interact with the rest of the system, before selecting a real-time priority. To aid in this decision, the priorities of the system threads should be considered. IRIX manages system threads to handle kernel tasks, such as paging and interrupts. System daemon threads execute between priority range _9_0 _a_n_d _1_0_9 inclusive, and system device driver interrupt threads execute between priority range _2_0_0 _a_n_d _2_3_9 inclusive (see the following section for more information about interrupt threads). An application may set the priorities of its threads above that of the system threads, but this may effect the behavior of the system. For example, if the disk interrupt thread is blocked by higher priority user thread, disk data access will be delayed, pending completion of the user thread. Setting the priorities of application threads within or above the system thread ranges requires an advanced understanding of IRIX system threads and their priorities. The priorities of the IRIX system threads may be found in /_v_a_r/_s_y_s_g_e_n/_m_t_u_n_e/_k_e_r_n_e_l. If necessary, these defaults may be changed using _s_y_s_t_u_n_e(1M), although this is not recommended for most users. Many soft real-time applications simply need to execute ahead of timeshare applications, in which case priority range 0 through and including 89 is best suited. Since timeshare applications are not priority scheduled, a thread running at the lowest real-time priority (0) will still execute ahead of all timeshare applications. Note, however, that at times the operating system briefly promotes timeshare threads into the realtime band to handle timeouts, and avoid priority inversion. In these special cases, the promoted thread's real-time priority is never boosted higher than 1. Applications cannot depend on system services if they are running ahead of the system, without observing the system responsiveness timing guidelines below. Interactive real-time applications (such as digital media) need low latency response times from the operating system, but changing interrupt thread behavior is undesirable. In this case, priority range 110 through and including 199 is best suited, allowing execution ahead of system daemons but behind interrupt threads. Applications in this range are PPPPaaaaggggeeee 4444 rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) typically cooperating with a device driver, in which case, the correct priority for the application is the priority of the device driver interrupt thread minus 50 (see the following section). If the application is multi-threaded, and multiple priorities are warranted, then the priorities of the threads should be no greater than the priority of the device driver interrupt thread minus 50. Note that threads running at a higher priority than system daemon threads should never run for more than a few milliseconds at a time, in order to preserve system responsiveness. Hard real-time applications may use priorities 240 through and including 254 for the most deterministic behavior and the lowest latencies. However, if a thread running at this priority ever gets into a state where it is using 100% of the processor, the system may become completely unresponsive. Threads running at a higher priority than the interrupt threads should never run for more that a few hundred microseconds at a time, in order to preserve system responsiveness. Priority 255, the highest real-time priority, should not be used by applications. This priority is reserved for system use in order to handle timers for urgent real-time applications, and kernel debugger interrupts. Applications executing at this priority run the risk of hanging the system. The proprietary IRIX interface for selecting a real-time priority, _s_c_h_e_d_c_t_l(), is still supported for binary compatibility, but it is no longer the interface of choice. The non-degrading real-time priority range of schedctl() is re-mapped onto the POSIX real-time priority band as priorities 90 through 118 as follows: 39=90, 38=110, 37=111, 36=112, 35=113, 34=114, etc.. Note that the large gap between the first two priorities preserves the scheduling semantics of schedctl() threads and system daemons. Real-time users are encouraged to use tools such as par(1) and irixview(1) to observe the actual priorities and dynamic behaviors of all threads on a running system. _DDDD_eeee_vvvv_iiii_cccc_eeee _DDDD_rrrr_iiii_vvvv_eeee_rrrr _IIII_nnnn_tttt_eeee_rrrr_rrrr_uuuu_pppp_tttt _TTTT_hhhh_rrrr_eeee_aaaa_dddd _PPPP_rrrr_iiii_oooo_rrrr_iiii_tttt_iiii_eeee_ssss As of IRIX 6.4, device drivers employ interrupt threads to handle device interrupts. Interrupt threads have default priorities in the range 200 through and including 239. To make selecting an appropriate priority for an interrupt thread easier, IRIX defines device classes including: audio, video, network, disk, serial, parallel, tape, external. Each device class has a priority assigned to it. A complete listing of device classes, and their default priorities, can be found in /_v_a_r/_s_y_s_g_e_n/_m_t_u_n_e/_k_e_r_n_e_l. For example, the value of _n_e_t_w_o_r_k__i_n_t_r__p_r_i defines the interrupt thread priority of all network class devices. PPPPaaaaggggeeee 5555 rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) A device driver may set the priority of its interrupt thread to one of the defined classes, by using the class directive in its driver configuration file (located in the /var/sysgen/master.d directory). For example, /var/sysgen/master.d/if_ef includes the directive +thread_class network which means that the value of the systune(1M) variable _n_e_t_w_o_r_k__i_n_t_r__p_r_i will be used for the interrupt thread priority of this device. Devices whose class cannot be determined use the value of the variable _d_e_f_a_u_l_t__i_n_t_r__p_r_i: +thread_class default The default priority of each device class may be changed using the appropriate systune(1M) variable in /_v_a_r/_s_y_s_g_e_n/_m_t_u_n_e/_k_e_r_n_e_l. The thread_class value may be overridden for a particular driver by adding the _t_h_r_e_a_d__p_r_i_o_r_i_t_y directive to the driver description file. For example: +thread_priority 205 On systems supporting the hardware graph, both of these values may be overridden for a particular device by using the DEVICE_ADMIN directive with the INTR_SWLEVEL attribute in the /var/sysgen/system/irix.sm file (q.v. for an example of this usage). _PPPP_rrrr_oooo_cccc_eeee_ssss_ssss_oooo_rrrr _CCCC_oooo_nnnn_tttt_rrrr_oooo_llll Using the _s_y_s_m_p() call or the _m_p_a_d_m_i_n and _r_u_n_o_n commands a programmer may control the distribution of processes among the processors in a real-time system. For instance, it is possible to bind a particular process onto a processor and conversely, it is possible to restrict a processor to only run those processes that are explicitly bound to it. This makes it possible to dedicate one or more processors to particular processes. Nominally, when IRIX is running in a multiprocessor certain system services require synchronization of all processors in the complex. This is mainly done to synchronize the instruction caches and to synchronize the virtual to physical translation caches or tlbs. In order to reduce the worst case dispatch latency a processor can be _i_s_o_l_a_t_e_d using the _s_y_s_m_p() call. This allows a process some control over when these synchronizing events take place. If the process never requests system services then there is no need to synchronize. If the process is sharing address space with other processes through use of either _s_p_r_o_c() or _s_p_r_o_c_s_p() then members of the share group should also avoid operations that would require IRIX to synchronize with the isolated processor. These include operations that explicitly flush caches, expand address space across 4 megabyte boundaries, release address space or change PPPPaaaaggggeeee 6666 rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) address space protections. Creation of new share group members through the use of _s_p_r_o_c() requires the creation of a stack area which may result in a synchronization event. Use of the _s_p_r_o_c_s_p() interface specifying a stack in section of locked memory is recommended. _s_y_s_m_p() can also be used to turn off normal IRIX clock processing on a particular processor and thus normal IRIX time slicing will not preempt the running process. Thus, if a processor is isolated, no devices are configured onto that processor, the clock service is disabled, the application process is restricted to the isolated processor and its virtual space is locked in memory then a user can achieve a fast bounded response time to an external event. _FFFF_rrrr_aaaa_mmmm_eeee _RRRR_aaaa_tttt_eeee _SSSS_cccc_hhhh_eeee_dddd_uuuu_llll_eeee_rrrr The Frame Rate Scheduler (FRS) is part of SGI's REACT/Pro product and is a special scheduling mechanism. It allows real-time processes and threads to have CPUs run them for specific amounts of time, in a specific order. The FRS is very useful for real-time applications that need a high degree of control over the CPU on which they are running, and which need to be scheduled at a specific frequency. _CCCC_PPPP_UUUU_ssss_eeee_tttt_ssss A cpuset is a named set of CPUs, which may be defined to be restricted or open. A restricted cpuset only allows processes that are members of the cpuset to run on the set of CPUs. An open cpuset allows any process to run on its cpus, but a process that is a member of the cpuset can only run on the CPUs belonging to the cpuset. Cpusets are useful for real-time applications because as sets of restricted CPUs they eliminate the interference caused by other processes running on the same CPU, while being more convenient to manage then individual restricted CPUs. _IIII_nnnn_tttt_eeee_rrrr_rrrr_uuuu_pppp_tttt _RRRR_eeee_dddd_iiii_rrrr_eeee_cccc_tttt_iiii_oooo_nnnn When the multi-processor real-time strategy is being used, it is often necessary to redirect unwanted PCI and VME interrupts away from the real-time processors. Control over which device interrupts are sent to which processor can be achieved by adding DEVICE_ADMIN directives to the /_v_a_r/_s_y_s_g_e_n/_s_y_s_t_e_m/_i_r_i_x._s_m file. The NOINTR directive may also be used to guarantee that no interrupts are randomly assigned for handling by the real-time processor. After _i_r_i_x._s_m is modified _l_b_o_o_t should be run to reconfigure the system. _KKKK_eeee_rrrr_nnnn_eeee_llll _TTTT_hhhh_rrrr_eeee_aaaa_dddd _PPPP_llll_aaaa_cccc_eeee_mmmm_eeee_nnnn_tttt PPPPaaaaggggeeee 7777 rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) rrrreeeeaaaallllttttiiiimmmmeeee((((5555)))) In some situations kernel threads need to run on specific processors or with other special behavior just like user threads. The XThread Control Interface (XTCI) was added in IRIX 6.5.16 to control these special behaviors. Users may add XTHREAD entries in the /_v_a_r/_s_y_s_g_e_n/_s_y_s_t_e_m/_i_r_i_x._s_m file. Kernel theads not mentioned operate with default behavior. After _i_r_i_x._s_m is modified _l_b_o_o_t should be run to reconfigure the system. To preserve compatability, XTCI entries will defer to the legacy /_v_a_r/_s_y_s_g_e_n/_m_a_s_t_e_r._d/_s_g_i interface in the event that conflicting entries are found. As in the _m_a_s_t_e_r._d/_s_g_i interface, system threads can also be specified but they may later change their behavior whereas interrupt threads must adhere throughout their lifetime. Specific interface entries include: XTHREAD name[*] [BOOT] [FLOAT] [STACK s] [PRI p] [CPU m...n] XTHREAD - Any line beginning with XTHREAD will be for controlling kernel threads. All the information must be on the same line. name[*] - Any thread with a name equal to "name" will be affected by the following directives. If [*] follows, any thread whose name begins with "name" will be affected. BOOT - The thread will stay within the boot cpuset if one exists. FLOAT - The thread will never be bound to a cpu. A thread cannot be both FLOAT & BIND. STACK - The number following STACK will specify the starting thread stack size. PRI - The number following PRI wll specify the starting thread cpu scheduling priority. CPU - The numbers following CPU will be a list of cpus to attempt to place the thread on if possible. Threads that cannot be placed on their CPU list will be considered FLOAT. This is comparable to the _s_y_s_m_p() MP_MUSTRUN command for user threads. SSSSEEEEEEEE AAAALLLLSSSSOOOO lboot(1), mpadmin(1), runon(1), systune(1M), mlockall(3c), mpin(2), munpin(2), plock(2), sched_setparam(2), sched_setscheduler(2), sproc(2), sysmp(2), syssgi(2), aio_error(3), aio_read(3), aio_return(3), aio_write(3), lio_listio(3), system(4), signal(5), sigqueue(3) timer_create(3c), pthread(3p) nice(1), renice(1m), frs(3), cpuset(5) PPPPaaaaggggeeee 8888